Skip to content

fix: pass through max tokens for non-mantle Bedrock models#1719

Merged
tejaskash merged 3 commits into
aws:mainfrom
Tommypop2:fix-pass-max-tokens
Jul 10, 2026
Merged

fix: pass through max tokens for non-mantle Bedrock models#1719
tejaskash merged 3 commits into
aws:mainfrom
Tommypop2:fix-pass-max-tokens

Conversation

@Tommypop2

@Tommypop2 Tommypop2 commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Description

Ensures that the maxTokens entry in the configuration is passed through to the agent

Related Issue

Closes #1718

Documentation PR

N/A

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation update
  • Other (please describe):

Testing

How have you tested the change?

  • I ran npm run test:unit and npm run test:integ -> 1 failure but this also failed when I ran npm run test on the current master
  • I ran npm run typecheck
  • I ran npm run lint
  • If I modified src/assets/, I ran npm run test:update-snapshots and committed the updated snapshots

Checklist

  • I have read the CONTRIBUTING document
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the
terms of your choice.

@Tommypop2 Tommypop2 requested a review from a team July 9, 2026 13:25
@github-actions github-actions Bot added the size/s PR size: S label Jul 9, 2026
@Tommypop2 Tommypop2 changed the title fix: Pass through max tokens for non-mantle Bedrock models fix: pass through max tokens for non-mantle Bedrock models Jul 9, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Jul 9, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Jul 9, 2026
@github-actions github-actions Bot added size/s PR size: S and removed size/s PR size: S labels Jul 9, 2026
@tejaskash

tejaskash commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Hi @Tommypop2 , the export fix is correct - the exported load.py dropped model.maxTokens for non-Mantle BedrockModel, and this threads it through.

But I traced #1718 through the managed-harness path and I don't think this PR closes it. The issue's repro is agentcor e dev → "Max output tokens" in Harness Settings, and that field never touches load.py. load.py only exists on the agentcore export → code-based-agent path (export converts a managed harness into a standalone Python agent you deploy yourself). A managed harness interprets its model config server-side, it never renders load.py.

On the managed path, model.maxTokens is already wired end to end.

So the "field is empty" symptom isn't reproduced by the current code on the managed path. Before merging as "Closes #1718", can you confirm:

  1. Did you repro via agentcore dev (managed harness) or export + deploy (code-based agent)?
  2. What CLI / agent-inspector version?
  3. Was the value under model.maxTokens, not the top-level execution maxTokens, in harness.json?

If it's a managed-harness issue it's likely either an older UI version or server-side (harness service not honoring BedrockModelConfig.MaxTokens), and should stay open.

@agentcore-devx-automation agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Jul 9, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Jul 9, 2026
@Tommypop2

Tommypop2 commented Jul 9, 2026

Copy link
Copy Markdown
Contributor Author

Hi!
I encounted and tested with agentcore dev as that's where I initially found the error. Happy to test more on a deployed agent to verify the fix works for both. (Though I'd appreciate some guidance on reliably using enough tokens as consistently getting the model to error even with the lower token limit was sometimes unreliable even in dev)
The value was under model.maxTokens as follows:

"model": {
    "provider": "bedrock",
    "modelId": "global.anthropic.claude-sonnet-5",
    "maxTokens": 32000
  },

The changes did seem to fix the issue, though I guess that can't be true if the agentcore dev flow doesn't use load.py. I am creating the agents in typescript (rather than the declarative harnesses), but keeping the harness.json so the CLI commands can still work.

I'm on CLI version v0.23.0.

Thanks!

@Tommypop2

Tommypop2 commented Jul 10, 2026

Copy link
Copy Markdown
Contributor Author

Done some more testing:
When I was testing, I was still running agentcore deploy -y so I'm assuming that's what meant that the maxTokens in dev was updated.
I have a fix that I've tested to actually update the maxTokens value in dev without deployment (tested by setting a small maxTokens value then prompting and looking for token overflow). I've added it to this PR for now, but happy to split them into separate PRs and issues if needed

Thanks :)

@github-actions github-actions Bot added size/m PR size: M and removed size/s PR size: S labels Jul 10, 2026

@tejaskash tejaskash left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, this is the right fix, and it matches what I traced. I'm approving and merging this PR, I'll send a followup for the other model providers. Thank you for your contribution!

@agentcore-devx-automation agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Jul 10, 2026
@agentcore-devx-automation

Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Jul 10, 2026
@tejaskash tejaskash merged commit b6fc338 into aws:main Jul 10, 2026
34 checks passed
tejaskash added a commit that referenced this pull request Jul 10, 2026
…ck (#1729)

* fix(dev): apply model config for all providers in dev, not just Bedrock

#1719 defaulted the harness invoke model from the local spec so agentcore.json
model settings (notably maxTokens) apply in dev without a redeploy — but only
for Bedrock. OpenAI/Gemini/LiteLLM harnesses still silently used the deployed
model config until redeploy.

Extract buildModelOverride() to map every provider to its own invoke config
key (bedrockModelConfig/openAiModelConfig/geminiModelConfig/liteLlmModelConfig),
carrying only the fields each config accepts. Update tests to cover all four.

* docs: trim buildModelOverride comment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Setting maxTokens in harness.json does nothing on non-mantle models

2 participants